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Abstract We consider a class of inexact Newton regularization methods for solving 
nonlinear inverse problems in Hilbert scales. Under certain conditions we obtain the 
order optimal convergence rate result. 



1 Introduction 

In this paper we consider the nonlinear inverse problems 

F(x)=y, (1.1) 

where F : D(F) C X n- Y is a nonlinear Frechet differentiable operator between 
two Hilbert spaces X and Y whose norms and inner products are denoted as || ■ | 
and (•, •) respectively. We assume that (|l.lj) has a solution x> in the domain D(F) of 
F, i.e. F(x') = y. We use F'(x) to denote the Frechet derivative of F at x £ D(F) 
and F'(x)* the adjoint of F'(x). A characteristic property of such problems is their 
ill-posedness in the sense that their solutions do not depend continuously on the data. 
Let y s be the only available approximation of y satisfying 

\\y s ~y\\<S (1.2) 

with a given small noise level 8 > 0. Due to the ill-posedness, the regularization 
techniques should be employed to produce from y s a stable approximate solution of 

CD. 

Many regularization methods have been considered in the last two decades. In 
particular, the nonlinear Landweber iteration [5], the Levenberg-Marquardt method 
4,9 , and the exponential Euler iteration [7] have been applied to solve nonlinear 
inverse problems. These methods take the form 

x n+1 = x n - g an (F'(x n )*F'(x n )) F'{x n )* (F(x n ) - y s ) , (1.3) 
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where xq is an initial guess of x\ {ct n } is a sequence of positive numbers, and {g a } 
is a family of spectral filter functions. The scheme (II ,3|) can be derived by applying 
the linear regularization method defined by {g a } to the equation 

F'(x n )(x-x n ) = y s -F(x n ). (1.4) 

which follows from by replacing y by y and -F(x) by its linearization F{x n ) + 
F'(x n )(x — x n ) at x n . It is easy to see that 

ffrn) - V s + F'(x n )(x n+1 - z n ) = r an (F'(x n )F'(x n )*)(F(x n ) - 

where 

r a (A) = 1 - Xg a (X) (1.5) 

which is called the residual function associated with g a . For well-posed problems 

where F'(x n ) is invertible, usually one has \\r an (F'(x n )F'(x n )*)\\ < < 1 and 
consequently 

\\F{x n )-y s + F\x n ){x n+1 ~x n )\\ <H n \\F{x n )-y & \\. (1.6) 

Thus the methods belong to the class of inexact Newton methods [2J. For ill-posed 
problems, however, there only holds \\r an (F' (x n )F' (x n )*)\\ < 1 in general. In [3] the 
Levenberg-Marquardt scheme was considered with {a n } chosen adaptively so that 
(11.61) holds and the discrepancy principle was used to terminate the iteration. The 
order optimal convergence rates were derived recently in [5]. The general methods 
(|1.3p with {a n } chosen adaptively to satisfy (|1.6[) were considered later in [H1ITT] , but 
only sub-optimal convergence rates were derived in [15] and the convergence analysis 
is far from complete. On the other hand, one may consider the method (11.31) with 
{a n } given a priori. This has been done for the Levenberg-Marquardt method in [5] 
and the exponential Euler method in [7] for instance. 

In this paper we will consider the inexact Newton methods in Hilbert scales which 
are more general than (| 1 .3|) . Let L be a densely defined self-adjoint strictly positive 
linear operator in X. For each r € R, we define X r to be the completion of r)^ D(L k ) 
with respect to the Hilbert space norm 

\\x\\ r := \\L r x\\. 

This family of Hilbert spaces (X r ) rg R is called the Hilbert scales generated by L. Let 
xq € D(F) be an initial guess of x* . The inexact Newton method in Hilbert scales 
defines the iterates {x n } by 

x n +i = x n - g an (L- 2s F'(x n )*F'(x n )) L- 2s F'(x n y{F(x n ) - y s ), (1.7) 

where s € R is a given number to be specified later, and {a n } is an a priori given 
sequence of positive numbers with suitable properties. We will terminate the iteration 
by the discrepancy principle 

\\F(x ns )-y s \\ <t5< \\F(x n ) - y% < n < n 5 (1.8) 

with a given number r > 1 and consider the approximation property of x ns to a;' as 
6—^0. We will establish for a large class of spectral filter functions {g a } the order 
optimal convergence rates for the method defined by (jl.7l) and (jl.8l) . 

Regularization in Hilbert scales has been introduced in 1121 for the linear Tikhonov 
regularization with the major aim to prevent the saturation effect. Such technique has 
been extended in various ways, in particular, a general class of regularization meth- 
ods in Hilbert scales has been considered in |16j with the regularization parameter 
chosen by the Morozov's discrepancy principle. Regularization in Hilbert scales have 
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also been applied for solving nonlinear ill-posed problems. The nonlinear Tikhonov 
regularization in Hilbert scales has been considered in [10,3 , a general continuous 
regularization scheme for nonlinear problems in Hilbert scales has been considered in 
[17] . the general iteratively regularized Gauss- Newton methods in Hilbert scales has 
been considered in [8], and the nonlinear Landweber iteration in Hilbert scales has 
been considered in [15] . 

This paper is organized as follows. In Section 2 we first briefly review the relevant 
properties of Hilbert scales, and then formulate the necessary condition on {a„}, 
{g a } and F together with some crucial consequences. In Section 3 we obtain the 
main result concerning the order optimal convergence property of the method given 
by (jl.7p and fj 1 ,8|) . Finally we present in Section 4 several examples of the method 
(11.71) for which {g a } satisfies the technical conditions in Section 2. 



2 Assumptions 

We first briefly review the relevant properties of the Hilbert scales (X r ) rG R generated 
by a densely defined self-adjoint strictly positive linear operator L in X, see [3]. 
It is well known that X r is densely and continuously embedded into X q for any 
— oo < q < r < oo, i.e. 

\M q < O^WxWr, xeX r , (2.1) 

where 6 > is a constant such that 

\\x\\ 2 <9(Lx,x), x € D(L). (2.2) 

Moreover there holds the important interpolation inequality, i.e. for any — oo < p < 
q < r < oo there holds for any x € X r that 

IM|,<||:c|||^||:b||". (2.3) 

Let T : X H> Y be a bounded linear operator satisfying 

m||/i||-„<||Tft||<M||ft||_ 0) heX 

for some constants M > m > and a > 0. Then the operator A := TL~ S : X H> Y is 
bounded for s > —a and the adjoint of A is given by A* = L~ S T*, where T* : Y H> X 
is the adjoint of T. Moreover, for any \v\ < 1 there hold 

R{{A*AY' 2 ) = X v{a+s) (2.4) 

and 

c(v)\\h\\_ u[a+s) < \\(A*AY' 2 h\\ < c{v)\\h\\_ v{a+s) (2.5) 

on D((A* A) v / 2 ), where 

c(y) := minjm 1 ', M v } and c(y) = max{m",M 1 '}. 

If g : [0, ||A|| 2 ] i— > R is a continuous function, then 

g{A*A)L s = L s g{L- 2s T*T). (2.6) 

In order to carry out the convergence analysis on the method defined by (|1.7p and 
(|1.8p . we need to impose suitable conditions on {a n }, {g a } and F. For the sequence 
{a n } of positive numbers, we set 

n 

8-i=0, s„:=V— , n = 0,l,---. (2.7) 

* — ' OLi 

j=0 3 
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We will assume that there are constants cq > 1 and c\ > such that 

lim s n = oo, Sn+i < co^n and < a n < c\, n = 0, • . (2.£ 



We will also assume that, for each a > 0, the function g a is defined on [0, 1] and 
satisfies the following structure condition, where C denotes the complex plane. 

Assumption 1 For each a > 0, the function 

1 



a + X 



extends to a complex analytic function defined on a domain D a C C such that [0, f ] C 
D a , and there is a contour r a C D a enclosing [0, 1] such that 

\z\>\a and p + ^ < b , \fz £ r a , a > and X £ [0, 1], (2.9) 

where bo is a constant independent of a > 0. Moreover, there is a constant b\ such 
that 

\<p a (z)\ \dz\ < bi (2.10) 



for all < a < c\. 

By using the spectral integrals for self-adjoint operators, it follows easily from 
(|2.9p in Assumption [T] that for any bounded linear operator A with ||A|| < 1 there 
holds 

\\( z I~A*A)-HA*Ay\\<j^L__ (2.11) 

for z £ r a and < v < 1. 

Moreover, since Assumption Q] implies (^(z) is analytic in D a for each a > 0, 
there holds the Riesz-Dunford formula (see pQ) 

ip a (A*A) = / <p a (z)(zl - A*A)~ 1 dz 



2m _ 

for any linear operator A satisfying ||A|| < 1. 

Assumption 2 Let {a ra } 6e a sequence of positive numbers, let {s n } be defined by 
\2. 7| ). There is a constant 62 > such that 

n 

< A" r Qfc (A) < (s n - Sj-i)-", (2.12) 
k=j 

0<A^ a ,(A) Yl r ak (X)<b 2 — (s n -s j - l )- v (2.13) 

for < f < 1, < A < 1 and j = 0, 1, • • • , n, where r a (X) is defined by U.5\ ). 

In Section 0] we will give several important examples of {g a } satisfying Assump- 
tions Q] and [2 These examples of {g a } include the ones arising from (iterated) 
Tikhonov regularization, asymptotical regularization, Landweber iteration and Lardy 
method. 

Lemma 1 The inequality implies for < v < 1 and a > that 

n 

O^X^a + X)- 1 J] r ak (X) < 2a"- 1 {l + a( Sn - S] ))- v (2.14) 
/or aZZ < A < 1 and j = 0, 1, • • • , rt. 
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Proof For < v < 1 and a > it follows from (|2~T2"j) that 

n 

0<A 1/ (a + A)- 1 J} r Qfc (A) <min{a ly - 1 ,a- 1 (s n -s 3 )- ,y } 

= a 1 ' -1 min {l, a~ u (s n — Sj) u } 

for all < A < 1 and j = 0, 1, • • • , n. □ 
Assumption 3 (a) There exist constants a > and < m < M < oo smc/i i/iai 
m||/i||_ a < \\F'(x)h\\ < M\\h\\- a , heX 

for all x £ i? p (x t ). 

(b) F is properly scaled so that \\F' (x)L~ s \\x^y < min{l, y/ag} for all x G 
B p (x^), where s > —a. 

(c) There exist 0<f3<l, 0<b<a and Kq > such that 

\\F'( X y - F'(^y\\ Y ^ Xb < K \\x - x*f (2.15) 

for all x G B p (x^). 

The number a in condition (a) can be interpreted as the degree of ill-posedness of 
F'{x) for x € B p {x s ). When F satisfies the condition 

F'(x) — R x F'{x^) and \\I - R x \\ < K \\x - x*\\, (2.16) 

which has been verified in [5] for several nonlinear inverse problems, condition (a) is 
equivalent to 

JT#||-a < \\F'(x^)h\\ < M\\h\\- a , heX 

From (a) and (H) it follows for s > -a that \\F'(x)L- s \\ x ^y < M9 a+S for all 
x G B p (x' f ). Thus II-F'^L^IIx-s-y is uniformly bounded over B p (x^). By multiplying 
(11.11) by a sufficiently small number, we may assume that F is properly scaled so that 
condition (b) is satisfied. Furthermore, condition (a) implies that F'(x)* maps Y into 
X b for b < a and \\F'(x)* \\v^x b < M6 a - h for all x G B p {x^). Condition (c) says 
that F'(x)* is locally Holder continuous around x^ with exponent < (3 < 1 when 
considered as operators from Y to X b . It is equivalent to 

\\L b [F'( X y - F'(x^*]\\ Y ^ x < K \\x -xY, * e B p (x*) 

or 

\\[F'(x) - F'(x*)]L b \\ x ^ Y < K \\x -xY, x G B,(a;t). 

Condition (c) was used first in |13] for the convergence analysis of Landweber iteration 
in Hilbert scales. It is easy to see that when 6 = and f3 = 1, this is exactly the 
Lipschitz condition on F'(x). When F satisfies (12.16[) . (c) holds with b = a and (3 = 1. 
In [T3] it has been shown that (c) implies 

\\F(x) -y- F'(x^)(x - 2^)11 < K Q \\x - x^f\\x - x*\\- h (2.17) 

which follows easily from the identity 

F{x) -y- F'(x^)(x - xt) = f [F'(x* + t(x - it)) _ F'(x^)] L h L~\x - x*)dt. 

Jo 



6 



In this paper we will derive, under the above assumptions on {«„}, {g a } and 
F, the rate of convergence of x ns to x^ as S — >• when eo := xq — x> satisfies the 
smoothness condition 



xo — x^ G with — — — < jj, < b + 2s, 



(2.18) 



where ns is the integer determined by the discrepancy principle (|1.8[) with r > 1. 

The following consequence of the above assumptions on i* 1 and {<?q,} plays a crucial 
role in the convergence analysis. 

Lemma 2 Let {g a } satisfy Assumptions^ and\^ let F satisfy Assumption^ and let 
{a n } be a sequence of positive numbers. Let A — F'{x^)L~ s and for any x £ B p (x') 
let A x = F'(x)L- s . Then for -57^7 < v < 1/2 there holds^ 



(A* Af J] r ak (A*A)[g aj (A*A)A*-g aj (A* x A x )A*] 

< _Lf _ o. ir-iu _ 



/or j = 0, 1,-- - ,n. 

Proof Let ?7a(A) = (a + A) -1 and </? Q (A) = g Q (A) — (a + A) -1 . We can write 

n 

(A* A)" [] r Qfc (A* A) [g aj {A* A) A* - g aj (A* X A X )A X ] = J t + J 2 + J 3 , 
fc=j+i 

where 



J x :=(A*Ar H r a „(A*A)g aj (A*A)[A*-AZ 



k=j+i 



J 2 := {A* AY 11 r ak (A*A) [r) aj (A* A) - V*M*M] A l> 
k=j+i 

n 

,h := {A* AY J] ^ (A* A) [<p aj (A* A) - Voij (A* X A X )} A*. 
fe=i+i 

It suffices to show that the desired estimates hold for the norms of J%, Ji and J3. 
From (|2.5p , (I2.13[) in Assumption [5] and Assumption [3] it follows that 



I Jill < 



{A* AY J] r ak (A*A)g aj (A*A)(A*A)^ 
k=j+l 



(A*A)-5&+7)[A* x -A*] 



< sup A" + ^t 5qj (A) ^(A) ]\\L b [F'(xr-F'(xiy]\\ Y ^ x 
\ k= J+ i ' 



1 Throughout this paper we will always use C to denote a generic constant independent of <5 and 
n. We will also use the convention <P < to mean that ^ < CP for some generic constant C. 
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which is the desired estimate. 

In order to estimate || J2II, we note that 

Vaj (A* A) - Vctj (A* X A X ) = fal + A*A)- 1 A*(A X - A)(ajl + A^y 1 

+ + A*A)-\A* X - A*)A x ( aj I + A* x A x y\ 

Therefore J2 — J% + where 

n 

j£ ] = {A*AY JT r ak (A*A)(a 1 I + A*A)- 1 A*(A x -A)(a 1 I + A x A x y 1 A* x7 
fe=i+i 

n 

4 2) = (A*AY JT r ak (A*A)(a j I + A*Ay 1 (A x -A*)A x A* x (a j I + A x A*y\ 
fe=i+i 

With the help of Assumption [3] and (|2.5I) we have for any w £ Y that 
||(^ x -A)(a i / + ^)- 1 A>|| 

= ||[r(^)-F'(xt)]L b J L-^)(a j /+4^ x )-M>|| 

<K \\x-y\f Wiajl + AlA^AlwW^+s) 

< K \\x - y\f\\(A* x A x )^( aj I + A%A x )^A* x w\\ 

<K Q \\x-x^fa: k+ ^\\w\\. 

This implies 

\\(A X - A)(a 3 I + A* x A x y'A*J < K \\x - xt||^ + ™. (2.19) 
Thus, by using Lemma [2 we derive 



|J 2 (1) || < sup [ A^+^^+A)- 1 JT r ak (X)\ UA^A^I + AIA^AH 

k=j+i 



0<A<1 



<«■ 1+2< ° +Sa) (l + a^Sn-sj^-iKoWx-x^f. 

By using Assumption [3J Lemma Q] and a similar argument in estimating J\ we can 
derive 

Il4 2) ||< sup A^fe+A)- 1 f[ r Qfe (A)| \\L b [F'(xy-F'(xiy]\\ Y ^ x 

< a ;~ 1+3 ^ (1 + Qi (s n - 8j ))- v -*&fc K \\x - xif. 

Combining the above estimates on J% and and noting 2 ^+ s ) < 5, it follows 
that 

Ikall < «r 1+ ™ (! + a j( s » ~ sj))-"-^ K \\x - y\f 
= — fa ~ s^ x y v -^ K Q \\x - yf . 

Otj 

It remains to estimate J3. Since Assumption [1] implies that <p aj (z) is analytic in 
D aj , we have from the Riesz-Dunford formula that 

J 3 = ^-r / ip a yz)Tj{z)dz, (2.20) 
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where 



T j {z):={A*AT J] r ah (A*A)[(zI-A*A)- 1 -(zI-AtA x )- 1 ]A:. 
k=j+i 



We can write Tj(z) = Tf ] {z) + T^'(z), where 

n 

Tf-\z):={A*A) v J] r ah (A*A)(zI-A*A)- 1 A*(A-A x )(zI-A* x A x )- 1 A* x , 



k=j+i 



Tj 2 \z):=(A*Ay Yl r ak (A*A)( Z I-A*Ay 1 (A*-A* x )A x A* x (zI-A x A:r 1 . 



We will estimate the norms of T^\z) and T^ z, (z) for z e r aj . With the help of 
Assumption 21 (12.51) and (I2.11[) . similar to the derivation of (|2.19p we have 

\\(A - A x )(zl - AIA^AIW < K \\x - a;tf ^-Hag^ . 

Since \z\ > otj/2 and \z - A| _1 < fo (|^| + A) -1 for z e r aj , we have from (|2.14p in 
Lemma [T] that 

11^(2)11 ^JfoHx-stf^-Hs^y sup lA^lz-Ar 1 f[ r ah (X) 



(2), 



0<A<1 



k=j+l 



<K \\x-xY\z\-*+^ mp^ ( A-+i(|^| + A)- 1 [J r Qfc (A) 



0<A<1 



< A ||z - xt||fl| 2! |>'-i+ 3 fi W (i + ( Sn _ Sj )| z |) 



-I/-1/2 



; *r ||z-zt||V; 1+2( " +s) (1 + (s n - Sj ) aj ) 



-v-l/2 



Next, by using (|2 . 14[) in Lemma [TJ f|2 . 5|) . Assumption [3j a) and (|2.11l) . we have for 
z e r aj that 



(A* A)^ J] r Q(c (A*A)(z/ — A* A)~ 1 (A* A) z 

k=j+i 



(A*Ay^^(A* - A* x )A x A* x {zI - A X A* X )~ 1 
< sup l\"+*£fr\ z -\\-i f[ r ak (X))\\L b (F\^r~F'(xr)\ 



0<A<1 



fc=i+i 

b+s 



<K \\x-x1f sup A^+^dzl+A)- 1 TT r ak (X) 
\ k= J+ i 

< K \\x - x*f \zr 1+ ^l(l + (s n - Sj )\ z \)-»-^ 



<K \\x-xifa 



ii^- 1+5 ^(i + ( Sn -^)^r-^y. 



Combining the above estimates on Tj(z) and T^ 2 \z) and noting 2 ut+s) — 2> ^ 



follows for z € r" Q . that 



||T,(z)|| < Aob - xtfa;~ 1+5T ^(l + (,s„ - Sj ) aj )-"-^ 

= ^(S n ~ S j . 1 )- V -^K \\X - X^f 
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Therefore, it follows from (12.20)) and Assumption [T] that 

\\J3\\<—(s n -s^ 1 )- u -^K \\x-x^f f \<p aj (z)\\dz\ 
a o Jr aj 

< —(Sn - 8 j - 1 )- V -*&fo K \\X - X^f. 
a j 

The proof is therefore complete. □ 



3 Convergence analysis 

We begin with the following lemma. 

Lemma 3 Let {a n } be a sequence of positive numbers satisfying a n < c\, and let s n 
be defined by \2. 7p . Let p > and q > be two numbers. Then we have 

n ( 1, maxjp, q} < 1, 

V — (s„ - 8 i -i)-'V < Cos 1 -*-" \ log(l + «„), max{p, q} = 1, 
j=o J { s n , maxjp, > 1 , 

where Cq is a constant depending only on C\, p and q. 

Proof This result is essentially contained in [SJ Lemma 4.3] and its proof. For com- 
pleteness, we include here the proof with a simplified argument. We first rewrite 



E -(*» - ^rv - ^ E — ( 1 - — ) ( ? 

j— o ^' j— o ' Sf ; ' S '' 

Observe that when < Sj-i/s n < 1/2 we have 



Sji J \ $n / \ S r , 

while when sj-i/s n > 1/2 we have 
Consequently there holds with C P:q — max{2 p , 2 9 } 

-P„-9 



J'=0 



j = a j s n V s ™/ 3=0 a J S ™ 



Note that Sj — Sj_i = 1/ofj, we have with h = 2a 1 s 

"1 " rsj/s n i-so/a 



/ = E / r9dt + / 

J so / s„ —h 3=1 Sji ~ 1 

n 

^E 

i=i 

1 I 



lso/sn—h j =1 Jsj-i/s n Js /s n -h 

Sj \ Sj - Sj-l t 1 ( s N 



.1. x — > i. / ; \ 

> 



j=0 J x 
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Therefore 

^ — < 2 / t" 9 ^ < { 21og(2a oSn ), 9 = 1, 

J=Q U 3 Sn\SnJ Js a / Sn -h [ _2_( 2aoS „)9-l 7 g > 1. 

By a similar argument we have with h = w— 



(3.2) 



2a n s n 



2 



i / „. t\~p r-^r+ h i-p' 
X!— 1_ (l-i)~ p dt< < 21og(2a„s„), p=l 



p < 1, 

'g(2a„s„), 

]=0 a j s n\ s n J J Q [JL.(2a n s n )i>-\p>l. 

(3.3) 

Combining p.l[) . (13.21) and (|3.3p and using the condition a n < c\, we obtain the 
desired inequalities. □ 

In order to derive the necessary estimates on x n — X s , we need some useful iden- 
tities. For simplicity of presentation, we set 

e„ := x n - x\ A;= F'(x^)L~ s and A n ;= F'(x n )L~ s . 

It follows from (JTT7J) and (|2~B1) that 

e n +i = e n - i -8 ^ «A n ) A;(F(a;„) - 

Let 

u„ := F{x n ) -y- F'(x^)(x n - x t ). 

Then we can write 

e n+1 = e„ - J L- s 5Q „(A*A)A*(F(a; Il ) - 

- L- s [.9a„(A;A„)A; - g a JA*A)A*} (F(x n ) - y s ) 
= L- s r an (A*A)L s e n - L- s g an (A*A)A*(y - y 5 + u n ) 

- L- s [g an (A n A n )A* n - g an (A* A) A*} (F(x n ) -y s ). (3.4) 

By telescoping f|3.4p we can obtain 

n 

e n+1 =L- s \{r a] {A*A)L s e 

0=0 

n n 

~ L ~'ll II r ctk (A*A)g aj (A*A)A*(y-y 5 + u j ) 

3=0 k=j+l 
n n 

- ^E II r - k {A*A) [g aj (A*Aj)A* - g aj (A*A)A*~\ (F( Xj ) - y s ). 

j=0 k=j+l 

(3.5) 

By multiplying (|33|) by T := F'(x r ) and noting that A = TL~ S and 

7T. 71 71 

7 e n r afc (^*) Pa .(^A*)^*=n^.(^*), 

i=Q fe=j+l 3=0 
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we can obtain 



Te n+ i - y 5 + y 

n n 

= A]J r aj (A*A)L s e + J] r U] (AA*)(y - y s ) 

n n 

"Ell r ak {AA*) 9a M A *)AA*u 3 

3=0 k=j+l 
n n 

E A II r ak (A*A)[g aj (A*A j )A*-g aj (A*A)A*](F(x j )-y s ). (3.6) 

j=0 k=j+l 

Based on (13.51) and (|3.6D we will derive the order optimal convergence rate of 
x ns to a;' when eo '■— xq — x^ satisfies the smoothness condition (|2.18[) . Under such 
condition we have L s eo € X^- s and |^rf | < 1. Thus, with the help of Assumption 
G3a), it follows from (|2.4I) and (12.51) that there exists w G X such that 



L s e = (AM) 5 *^^ and c 2 ||w|| < ||e || M < c 3 ||w| 



(3.7) 



for some generic constants C3 > C2 > 0. We will first derive the crucial estimates on 



and ||Te„||. To this end, we introduce the integer n$ satisfying 



(t-I)* 



"2(a + S ) 



< n < ns, 



(3.8) 



2co|M| 

where cq > 1 is the constant appearing in (I2.8[) . Such ri^ is well-defined since s„ — » 00 



asn-> 00. 



Proposition 1 Let F satisfy Assumptions^ let {g a } satisfy Assumptions^ and\^ 
and let {a n } be a sequence of positive numbers satisfying i2.8\) . If eo € X^ for some 
(a — b)/(3 < fi < b + 2s and if A"o||w||^ is suitably small, then there exists a generic 
constant C* > swc/i i/iaf 



||e n ||/i < C*IMI 



and 



\Te n \\ < C*s n 



2(a+ S ) 



and 



||Te„ - j/ 4 + y|| < (co + C* K \\uf)i r . 



(3.9) 
(3.10) 



for all < n < n$ . 

Proof We will show (|3.9p by induction. By using (|3.7[) and |jyl|| < -y/eio we have 

||Te || = \\AL s e \\ = \\(A*A)^ 2 L s eo\\ = \\(A* A)*&& w\\ < aj^||w||. 

This together with (|3.7[) shows (|3.9[) for n = if C* > max{l,C3}. Next we assume 
that (|3.9|) holds for all < n < I for some I < hs and we are going to show (|3.9p 
holds for n = I + 1. 

With the help of (j2~51) and (|3~71) we can derive from Q3.5P that 



l e /+ilU 



< 



U r °A A * A ) 



J2(AA*)^Tg*M A *) IT r ak (A A *)(y-v 5 + ^) 



k=j+l 



I I 
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Since (a — b) / f3 < fi < b + 2s and < b < a, we have 

a + 2s — n , b + s s — u 1 
0< — 7 < 1 and - , < — — — < — , 

2(a + s) 2(a + s) ~ 2(a + s) 2 

Thus we may use Assumption [5] and Lemma [2] to conclude 

; 



ih+iiu < imi + E -( s < - «i-ir^(s + kid 

+ E ^ - Sj-ir^KoWe^WF^) - y s \\. (3.11) 



Moreover, by using (|3.7I) . Assumption [5] and Lemma [51 we have from (|3.6p that 
||Te i+1 - v s + < s^^NI + ^(*J ~ *i-i) _1 |NI 



1 1 j 



! 

C 4 



E -( s ' - s^y^KoWejfWFixj) - y 5 \\, (3.12) 



a 

3=0 ■ 



where C4 > is a generic constant. 

By using the interpolation inequality (j2.3[) . Assumption j3ja) and the induction 
hypotheses, it follows for all < j < I that 



Nl < WejWlTWejWr < WTeiW^WejWf < \\ U \\ 8j 2( " +s) . (3.13) 
With the help of (|2.17[) and the interpolation inequality (|2.3[) , we have 

KH <K a \\ ej \f \\ ej \\. b < K \\ ej \\_^ Hej-IU^" . (3.14) 
We then obtain from Assumption^ a) and the induction hypotheses that 

INI <^o||Te i ||^+^|| ej -|| M - + " ^Xoll^H 1 ^;^^. (3.15) 
On the other hand, since (|2.1I) and the induction hypotheses implies 

INI-« SINUS IMI, 0<j<l 
and since \i > (a — &)//3, we have from (13.141) and Assumption 03a) that 

6-q+q/3 q+q/3-a 

||u i ||<lifo||e J ||_ a ||e i ||_r'' IMU^" < K \\ujf\\Te 3 \\. (3.16) 
Therefore, by using the fact 

5<^-\\u;\\sJ^, 0<j<l (3.17) 

T — 1 J 

and the induction hypotheses we have 

\\F( Xj ) y s \\ < S + \\TejW + \\ Uj \\ < \H\s~^ . (3.18) 
In view of the estimates (|3.13[) , (|3.15j) . ()3.18[) and the inequality 
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which follows from Lemma [U we have from (|3.1ip and (|3.12[) that 



||ej+ilU < C 5|MI +c 5 sp a+3) d 



1 1 n+9,-„ 



+ CX |M| 1+/3 E — ( s ' - s J --i)"" ajSTir a j 



a, 

— n J 



3=0 



and 



|Te i+1 - y 5 + y\\ < \\ u \\ 8l ^ + 6 

i 



b+,, + ,i;-> 

1 2(a+s) 

3=0 J 



— n 3 



a 

3=0 



where C5 and C are two positive generic constants. 

With the help of Lemma El fi> {a- b)/(3, (|3~TT|) and ([H} we have 

||e ;+ i|| M < ^c 5 + ^-yc c 5 + C^oll^H^^) |M|, 

and 

||Te m -y s + y\\ < S + (l + CJT ||wf ) IM!^™ 

< 5 + c (1 + <7#o|Ml") IMK+f^- (3-19) 

Consequently ||ej+i|| M < C*||o;|| if C* > 2cs + t^jCoCs and iiolMI' 3 is suitably small. 
Moreover, from ([335]) . (|3~T7| and ([2T5]l we also have 

||Te m || < 26 + co (1 + CK Q \\uf) |M| S ~f^ 



< 



4c; 



_ Sj + co + c-KoNI^IHI^+i' 



4c 2 

if C* > 2co + and iiolMI^ is suitably small. We therefore complete the proof of 
(13J1) . In the meanwhile, (1511)) gives the proof of (|3~T0| . □ 

From Proposition [T] and its proof it follows that x n £ B p {x [ ) for < n < ng if 
||w|| is sufficiently small. Furthermore, from (13. 15)) and (|3.16p we have 

_b± J± +j ± p 

\\F(x n )-y-Te n \\<K Q \\uj\\ 1+ ^ Sn 2( "+ s) (3.20) 

and 

\\F(x n ) -y- Te n \\ < K \\uf\\TeJ (3.21) 

for < n < h$. 

In the following we will show that ns < n$ for the integer n$ defined by (|1.8p with 
r > 1. Consequently, the method given by (|1.7p and (|1.8p is well-defined. 
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Lemma 4 Let all the conditions in Proposition^ hold. Let r > 1 be a given number. 
If eo £ for some (a — b)/f3 < /i < b + 2s and if Ko\\eo\\P is suitably small, then 
the discrepancy principle \1.8\) defines a finite integer n$ satisfying ng < h$. 

Proof From Proposition [TJ p.20j) and \i > (a — b)/f3 it follows for < n < hg that 

\\F(x n ) - y s \\ < \\F{x n ) -y- Te n \\ + \\Te n - y 5 + y\\ 

< CK \\u\\ 1+ P S ~ b ~^ L + (c + CK \\uf) s~^\\u>\\ + 5 

< (co + CK \\uf) Sn^M + S. 

By setting n — hg in the above inequality and using the definition of n$ we obtain 
\\F(x ns ) - /|| < (l + + CKoM^ S < rS 

if -Ko|M|^ is suitably small. According to the definition of ng we have ng < fig. □ 

Now we are ready to prove the main result concerning the order optimal conver- 
gence rates for the method defined by (|1.7p and (jl.8p with r > 1. 

Theorem 1 Let F satisfy Assumptions^ let {g a } satisfy Assumptions [7] and\^ 
and let {a n } be a sequence of positive numbers satisfying \2.8\ ). If cq £ X^ for some 
(a — b)/(3 < /i < b + 2s and i/i^o||eo||^ is suitably small, then for all r £ [— a, /i] there 
holds 

\\x ns -x^\\ r <C\\e \\fi5^ 

for the integer ng determined by the discrepancy principle i t _?.#)) with r > 1, where 
C > is a generic constant. 

Proof It follows from (|3.2ip that if fCo||w||' 3 is suitably small then 

\\F(x n ) - y - Te n \\ < ^\\Te n \\ 

which implies || Te rl | < 2\\F(x n ) — y\\ for < n < ng. Since Lemma 0] implies ng < fig, 
it follows from Assumption |3]Ja) and the definition of ng that 

IKJ-„ < -\\Ten s \\ < - (\\F(x ns ) - y s \\+6) < ^±^6. 
m m m 

But from Proposition [T] we have llenJI^ < C*||w||. The desired estimate then follows 
from the interpolation inequality (|2.3I) and (I3.7[) . □ 

Remark 1 If F satisfies (I2.16[) and {x n } is defined by (|1.7D with s > —a/2, then the 
order optimal convergence rate holds for xq — x^ £ X^ with < [i < a + 2s. On the 
other hand, if F'(x) satisfies the Lipschitz condition 

\\F'(x) - F'(x^)\\ < K \\x - x% x £ B p (x^) 

and {x n } is defined by (ll.7[) with s > a/2, then the order optimal convergence rate 
holds for xq — a;^ <G X^ with a < /i < 2s. 
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4 Examples 

In this section we will give several important examples of {g a } that satisfy Assump- 
tions Q] and [5] Thus, Theorem Q] applies to the corresponding methods if F satisfies 
Assumption [3] and {a n } satisfies (12.81) . For all these examples, the functions g a are 
analytic at least in the domain 

D a := {z e C : z ^ -a, -1}. 

Moreover, for each a > 0, we always take the closed contour r a to be (see pQ) 



with 



r (l) 


:={* 


2 ' 


00 < 4> < 27T - O }, 


r(2) 
a 


:={* 


= i?e^ : 


-00 < < <M, 


r(3) 

x a 




= te^ : 


a/2<t< R}, 


r( 4 ) 

a 


:={z 


= te-^° 


:a/2<t< R}, 



where R > max{l,a} and < <fio < ir/2 are fixed numbers. Clearly r a C D a and 
[0, 1] lies inside r a . It is straightforward to check that (|2.9p is satisfied. 

Example 1 We first consider for a > the function g a given by 

{a + X) N - a N 



5a (A) 



A(a + A) Ar 



where A > 1 is a fixed integer. This function arises from the iterated Tikhonov 
regularization of order A for linear ill-posed problems. The corresponding method 
(11.71) becomes 



^n,0 — ^n? 

u n ,i+i = u n j - (a n L 2s + T*T n ) 1 T* (F(x n ) - y 5 - T n (x n - u n j)) , 

1 = 0, ■■■ ,N-1, 

$n+l = u n,N, 

where T n := F'(x n ). When A = 1, this is the Levenberg-Marquardt method in 
Hilbert scales. The corresponding residual function is r a (X) = a N (a + \)~ N . In order 
to verify Assumption [21 we recall the inequality (see [9l Lemma 3] ) 

n 

A T-r _Ok , \-i for all A >0. 

a k + A 

fc—J 

Then for < v < 1 and A > we have 



A"II^(A)< A Il^fA ^(«»— 
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n f„ i \\N ff n / \ N 

W /n TT (\\ ( a J + A ) ~ < TT / «fc ' 
A 5a, (A) || r Qfc (A) = -—-j || 



and 



N-l / -\t\ n / \ N 

< C N — (s„ - s,,-_i)~ w , 

where Cn = 2^ — 1 and we used the fact aj 1 < s„ — Sj-\. We therefore obtain (|2.12j) 
and (|2.13[) in Assumption [5J 

Next we will verify (|2.10[) in Assumption [TJ Note that 



lfia[Z) ~ z(a + z) N z(a + z)"f^{ J 



It is easy to check |(^ Q (z)| < a 1 on 2"a and |^? a (z)| < 1 on r a 2 \ Moreover, on 
ri 3) U ri A) there holds 

N-2 JV-2 

v ; i=o j=o 



Therefore 



WOI|dz| = / M«)||<fe| + / W a {z)\\dz\ + I \<p a (z)\\dz\ 



< 



1 / ad(f>+ d(f)+^a 3+1 I t _2_J 'd< 

J 00 J—<t>o j—n J a/2 



< 1. 

Assumption [T] is therefore verified. 

Example 2 We consider the method (jl.7l) with g Q given by 

3a(A)^i(l-e- A /" 

which arises from the asymptotic regularization for linear ill-posed problems. In this 
method, the iterative sequence {x n } is equivalently defined as x n +i := x(l/a n ), where 
x(t) is the unique solution of the initial value problem 

j t x(i) = L- 2s F'(x n )* (y s - F(x n ) + F'{x n )(x n - x(t))) , t > 0, 
x(0) = x n . 
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The corresponding residual function is r a (A) = e x / a . We first verify Assumption [5] 
It is easy to see 

n 

A" [] ( A ) = X"e- x ^-'^ < v v e-"{s n - Sj . x )- V < (s n - s^)-" 

fc=j 

for < v < 1 and A > 0. This shows (I2.I2[) . By using the elementary inequality 
e~ pX — e~ qX < (q — p)/q for < p < q and A > and observing that < r Q (A) < I 
and < (? Q (A) < 1/a, we have for < v < 1 and A > that 

n / n \ 

A"ffa,(A) II r ah (X)< — \Xg aj W n r ^( A ) 
fe=i+l a J \ fc=i+l / 



-(s n ~ Sj )X _ -(s„-Sj_i)A 



< — (s„ - Sj-i) " 
a 3 

which gives (|2.13p . 

In order to verify (12 . 10[) in Assumption [TJ we note that 

. , l-e~ z / a 1 a - (a + z)er z l a 

<p a {z) = — = -, — • 

z a + z z{a + z) 

It is easy to see that |</3 Q (z)| < a^ 1 on r„ , \(p a (z)\ < 1 on r& and 

\<Pa{z)\ £ ... £ at 

t{a + 1) 

on ri 3) uri 4) . Therefore 

IVaWWdz] < 1+ / ai~ 2 dt < 1. 

T Q Ja/2 

Example 3 We consider for < a < 1 the function g a given by 

[1/Ql ' 1 1 - (1 - A)[V«1 



A 



which arises from the linear Landweber iteration, where [1/a] denotes the largest 
integer not greater than 1/a. The method (ll.7[) then becomes 

u„,i+i = u n ,i - L~ 2s T* (F(x n ) - V s ~ T n (x n - u ni i)) , < / < [l/a n ] - 1, 

a^n+l = Mn,[l/ „], 

where T„ := F'(x n ). When a„ = 1 for all n, this method reduces to the Landweber 
iteration in Hilbert scales proposed in [13]. The corresponding residual function is 
r a (X) = (1 — A)! 1 /"!. We first verify Assumption [3] when the sequence {a n } is given 
by a n = l/k n for some integers k n > 1. Then for < ^ < 1 and < A < 1 we have 

n 

A" II r «*( A ) = A "( 1 - ^Y^- 1 < - < (*„ - ai-x) - ". 
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We thus obtain (gTTg) . Observing that < r aj (A) < 1 and < g aj (X) < 1/ctj for 
< A < 1, we have 

n I n \" 

*W*) II ^W<— Ap Qj (A) n r Qfc (A) 
fc =J +i a i \ k=j+i J 

= ^j 7 ((l-A)--^-(l-A)--^- 1 ) v . 

Thus, (|2.13D follows from the elementary inequality t v — t q < (q — £>)/<? for < p < q 
and < t < 1. 

In order to verify (|2.10[) in Assumption [1] in the definition of r a we pick i? > 1 
and < 4>q < ir/2 such that R < 2cos<^>o- Note that 

</W = — = -, : — s • 

z a + z z(a + z) 

By using the fact (1 + a) 1 /" < e we can see 

\<Pa(z)\ < oT x {\ + a/2) 1 /" < a' 1 on f' 1 ). 

According to the choice of R and cf>o, we have 1 + R 2 — 2i?cos0o < 1. Thus 

a + (q + g)(l + R 2 - 2_Rcos fa)' 1 /*!/ 2 (2) 
R(R + a) 

Furthermore, on we have 

a + (a + t)(l + t 2 -2t cos o ) 1/(2q) 



l<PaO)l 



< 



<(q + t) 

Therefore 

a+(a + t)(l+t 2 ~2tcos(/)o) 1/{2a) , 

<p a (z) dz < 1 + / ^ A , — dt 

J a/2 H« + 1) 

f R/a 1 + (1 + t)(l + cr 2 * 2 - 2aicos ( /. ) 1 /(2«) 
= 1 + / 

Jl/2 t(l+t) 

rR/a 

<1+ {l + a 2 t 2 -2atcos(/)o) 1/(2a) dt. 

Jl/2 

Observe that for 1/2 < t < R/a there holds 

(1 + a 2 t 2 - 2crfcos0 o ) 1/(2a) < (1 - Moat) 1/(2a) < e"^*/ 2 
with ; — 2 cos (fro — R > 0. Thus 

|^a(*)||<fa| + / e-foWdt < 1. 

A/2 

Example 4 We consider for < a < 1 the function g Q given by 

i_(i + A)-[V«] 



.9. 



■ a (A)= £(l + A)-* = 
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which arises from the Lardy method for linear inverse problems. Then the method 
(|1.7I) becomes 

u n ,i+i = u n j - (L 2s + T^T^T* (F{x n ) - y s - T n (x n - u n>l )) , 

l = 0,--- ,[l/a n ]-l, 

X n +1 = U n> [i/ an ], 

where T n = F'(x n ). The residual function is r a (X) = (1 + A) - ! 1 /™!. Assumption Q] and 
Assumption [5] can be verified similarly as in Example [3] when the sequence {a n } is 
given by a n = l/k n for some integers k n > 1. 
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